Skip to content

Conversation

@SpencerLN
Copy link
Contributor

Proposed commit message

feat(filebeat): add file auth support for CEL and HTTP JSON inputs

  • Add auth.file configuration for CEL and HTTP JSON inputs to load header values from disk with optional prefix and refresh interval.
  • wrap client transports to inject the refreshed file-based credential on each request and guard against invalid configs

Note

The code is mostly duplicated between httpjson and CELinputs. I considered putting it in a shared location that each could reference, but I didn't see a clear pattern on how to do that. As it isn't much code I've implemented it in each input at the moment.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None. Existing configurations continue to work; the new file auth mechanism is opt-in.

Author's Checklist

  • [ ]

How to test this PR locally

  • Create a file (i.e. /etc/elastic/gh_token_test) containing an invalid secret
  • Start a Filebeat instance with a configuration that tries to retrieve data from a server requiring authentication:
filebeat.inputs:
  - type: httpjson
    interval: 5s
    auth.file:
      path: "/etc/elastic/gh_token_test"
      prefix: "token "
      refresh_interval: 10s
    request:
      url: https://api.github.com/repos/{org}/{repo}/issues
      method: GET
      transforms:
        - set:
            target: url.params.per_page
            value: "1"
        - set:
            target: url.params.state
            value: open
        - set:
            target: header.Accept
            value: application/vnd.github+json
  • Confirm that Filebeat logs show a 401 error
  • Update the secret file to contain a valid secret, wait 10-15 seconds, confirm the input now outputs the expected data

Related issues

Use cases

Filebeat reads short-lived (JWT, OAuth, etc.) tokens projected to disk by an external secret manager and refreshes them without restarting the input.

Note

GitHub Copilot was used to generate some code for this PR.

@SpencerLN SpencerLN requested review from a team as code owners November 6, 2025 23:37
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 6, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify
Copy link
Contributor

mergify bot commented Nov 6, 2025

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @SpencerLN? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

@SpencerLN SpencerLN changed the title Add file-based authentication support for CEL and httpjson inputs Add file-based authentication support for CEL and HTTP JSON inputs Nov 6, 2025
@SpencerLN SpencerLN added enhancement Team:Security-Service Integrations Security Service Integrations Team and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 6, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Team:Security-Service Integrations Security Service Integrations Team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Filebeat] [httpjson/cel] Support setting authentication headers based on file content

2 participants